S8.1 Policy sensitivity test: Scenario input conditions

Introduction

This document presents the results assessing the impact of input assumptions on policy behavior for the Decision Support tool for Child and Adolescence Obesity, a system dynamic model developed to explore underlying relationships that contribute to youth obesity.

Outline of behaviour sensitivity analysis

The test presented in this document aims to examine which input assumptions have the greatest impact on modeled policy behavior, unlike previous sensitivity analyses, which focus on numerical relationships between input assumptions and the effectiveness of policy scenarios.

Methods

A sensitivity analysis was conducted, testing 649 input variables between \(\pm\) 20% of an input’s initial value. While each input value may have more extreme values, these input values are considered to be outside of probable values. The model generates 78 BMI prevalence outcomes for each sensitivity analysis, covering 3 BMI categories, 13 age groups, and two genders.

A full depiction of the parameters tested can be found in the S5 Extreme conditions test: model parameters and S6 Extreme conditions test : policy parameters online supplementary materials.

Model behaviour in policy effectiveness is defined as the trend, phase, lag, or amplitude of a policy result over modelled time. Inputs that impact these outcome characteristics are considered to impact the model behaviour.

Outcome Measure

The behavior of each model test was evaluated using the cumulative difference of the first derivative. The following plots explain how this outcome measure is calculated.

Figure panel A shows a plot of three outcomes: the base model, where there are no changes to any initial assumptions; a result that varies an input assumption, producing similar policy behavior; and a second test where varying the input assumption produces different policy behavior. The absolute cumulative difference between the model outcomes is plotted in panel C.

Panel B plots the first derivative of each policy result. The result with similar behavior also has a similar first derivative. Conversely, the result with different behavior has a greater difference in the derivative. The cumulative absolute difference of the first derivative shows a greater difference between the two policy results, highlighting the distinction between behaviors.

While the cumulative absolute difference measure cannot provide insights into the model structure, it is useful to screen input assumptions that impact policy behaviour .

Unlike the extreme test, this analysis focuses on determining which input assumption impacts the policy behaviour.

Example plot for model behaviour

Example plot for model behaviour

How to interpret the results

The figure below breaks down the summary results shows in the summary plot and the figure above. Larger cumulative difference of the first derivatives reflect model runs that change the model patterned behaviour.

Panel A shows the summary plot for average cumulative difference in the first derivative when energy density of sugar is varied and each of the policy scenarios are implemented in the model. A higher positive value indicates model runs that have a greater impact on model behaviour. This suggests that changes to the assumed energy density for sugar has the greatest impact on the SSB tax policy scenario patterned behaviour.

This conclusion can be further investigated by examining Panel B, the reduction in obesity from policy. In this Panel, the patterned behaviour of the SSB column shows that peaks and troughs of the scenario effectiveness over time deviated from the initial input (dashed lines). This is also reflected in the first derivative plots in Panel C through large difference between sensitivity analysis model results and initial conditions. The cumulative difference between each sampled input and the initial condition first derivative is plotted in Panel D, the average is also plotted as a red line. The average at the end of the model corresponds to the summary plot in Panel A.

In the following result plots, only Panel A is presented; the overarching interpretation to note is

Larger cumulative difference of the first derivatives reflect model runs that change the model patterned behaviour

Results

Early Prevention of Obesity in Children ranking

Type Label Average cumulative difference in derivatives
Aggregate health effect Effect Non-core OR 1.12e-02
Aggregate exposure Percentage of infants exposed 9.78e-03
Aggregate health effect Effect BF OR 9.28e-03
Fidelity scale up EPOCH Scale up factor 8.60e-03
Aggregate health effect Effect TV OR 6.78e-03
Relapse Relapse Rate EPOCH 3.17e-03
Relapse Relapse Rate 2.22e-04
Economic assumptions Commonwealth Healthcare Costs 0.00e+00
Economic assumptions Early Childhood Prevention Cost 0.00e+00
Economic assumptions Household Healthcare Costs 0.00e+00
Economic assumptions Productivity Loss of guardians from kids missing school 0.00e+00
Economic assumptions State Healthcare Costs 0.00e+00
Economic assumptions $ per QALY 0.00e+00
Economic assumptions Invariant Cost proportions 0.00e+00
Economic assumptions Quality of Life Health Utilities per year Reference 0.00e+00
Note:
Ranked variables

Childcare setting intervention variable ranking

Type Label Average cumulative difference in derivatives
Fidelity scale up CCI Scale up factor 2.58e-02
Aggregate exposure Percentage of childcare centers participating in intervention 2.58e-02
Aggregate exposure “Percentage of 3-6 year olds attending formal childcare” 2.11e-02
Behaviour definition “Proportion of Fruit Juice in Non-alcoholic beverages” 1.52e-02
Aggregate health effect Childcare settings intervention Change in Diet coefficients 9.97e-03
Relapse Relapse Rate 9.94e-03
Behaviour definition Proportion of Cordial in SSB 8.78e-03
Behaviour definition Proportion of Packed snacks in Discretionary foods 6.25e-03
Behaviour definition “Proportion of Cakes/muffins/Cookies in Discretionary foods” 5.16e-03
Aggregate exposure “Percentage of 2-3 year olds attending formal childcare” 4.91e-03
Behaviour definition “Proportion of Chocolate/candy in Discretionary foods” 3.92e-03
Aggregate health effect Childcare settings intervention Change in PA coefficients 1.21e-03
Economic assumptions “Childcare-based Prevention Cost ($ per child /year)” 0.00e+00
Economic assumptions Commonwealth Healthcare Costs 0.00e+00
Economic assumptions Household Healthcare Costs 0.00e+00
Economic assumptions Productivity Loss of guardians from kids missing school 0.00e+00
Economic assumptions State Healthcare Costs 0.00e+00
Economic assumptions $ per QALY 0.00e+00
Economic assumptions Quality of Life Health Utilities per year Reference 0.00e+00
Note:
Ranked variables

School-based settings interventions ranking

Type Label Average cumulative difference in derivatives
Fidelity scale up SI Scale up factor 3.92e-02
Aggregate exposure SI Percentage of Schools participating in intervention 3.92e-02
Behaviour definition SI percentage of total diet consumed at school 2.97e-02
Relapse Relapse Rate 2.08e-02
Aggregate health effect SI % change in PA 9.26e-03
Aggregate exposure SI Percentage of children attending School 8.43e-03
Aggregate health effect SI % Reduction In student Purchases 2.72e-03
Economic assumptions Commonwealth Healthcare Costs 0.00e+00
Economic assumptions Household Healthcare Costs 0.00e+00
Economic assumptions Productivity Loss of guardians from kids missing school 0.00e+00
Economic assumptions “School-based Prevention Cost ($ per child /year)” 0.00e+00
Economic assumptions State Healthcare Costs 0.00e+00
Economic assumptions $ per QALY 0.00e+00
Economic assumptions Quality of Life Health Utilities per year Reference 0.00e+00
Note:
Ranked variables

Sports voucher interventions ranking

Type Label Average cumulative difference in derivatives
Fidelity scale up SVI Scale up factor 8.90e-02
Aggregate health effect Average proportion attributed to vouchers 8.90e-02
Behaviour definition Males Average MET Calculated Inputs 5.89e-02
Behaviour definition Females Average MET Calculated Inputs 4.54e-02
Relapse Relapse Rate 4.03e-02
Aggregate exposure “% of population that Registers for vouchers” 1.85e-02
Behaviour definition Average Number of Hours of organised sports or activities Weekly 6.70e-03
Economic assumptions Commonwealth Healthcare Costs 0.00e+00
Economic assumptions Household Healthcare Costs 0.00e+00
Economic assumptions Productivity Loss of guardians from kids missing school 0.00e+00
Economic assumptions “Sports Vouchers Program cost ($ per child /year)” 0.00e+00
Economic assumptions State Healthcare Costs 0.00e+00
Economic assumptions $ per QALY 0.00e+00
Economic assumptions Quality of Life Health Utilities per year Reference 0.00e+00
Note:
Ranked variables

Sugar-Sweetened Beverge Tax % ranking

Type Label Average cumulative difference in derivatives
Fidelity scale up SSB Scale up factor 1.46e-01
Aggregate health effect SSB Tax % 1.46e-01
Relapse Relapse Rate 7.60e-02
Behaviour definition PE Soft drink 1.74e-02
Behaviour definition PE Fruit Drink 9.42e-03
Behaviour definition PE Cordial 8.03e-03
Economic assumptions Commonwealth Healthcare Costs 0.00e+00
Economic assumptions Household Healthcare Costs 0.00e+00
Economic assumptions Productivity Loss of guardians from kids missing school 0.00e+00
Economic assumptions State Healthcare Costs 0.00e+00
Economic assumptions $ per QALY 0.00e+00
Economic assumptions Quality of Life Health Utilities per year Reference 0.00e+00
Economic assumptions Sugar Sweetened Beverage Tax cost 1st year 0.00e+00
Economic assumptions Sugar Sweetened Beverage Tax cost subsequent years 0.00e+00
Note:
Ranked variables

Overall Ranking of policy sensitivity

Average Cumulative difference
first derivative
CCI EPOCH SI SSB SVI
Fidelity scale up 0.026 0.009 0.039 0.146 0.089
Aggregate health effect 0.007 0.009 0.003 0.146 0.089
Relapse 0.010 0.002 0.021 0.076 0.040
Behaviour definition 0.008 0.030 0.010 0.012
Aggregate exposure 0.017 0.010 0.014 0.019
Economic assumptions 0.000 0.000 0.000 0.000 0.000
Note:
Average variable type

Early Prevention of Obesity in Children

Early Prevention of Obesity in Children Scale up of intervention effects

Early Prevention of Obesity in Children: RCT Intervention effect

Percentage of infants exposed to intervention

Early Prevention of Obesity in Children: Relapse Rate

Childcare setting intervention

Childcare setting intervention Scale up factor

Childcare settings intervention Change in Diet coefficients

Childcare settings intervention Change in physical activity coefficients

Percentage of childcare centers participating in intervention

Definition in food groups

School-based settings interventions

School-based interventions: Relapse Rate

Percentage of Schools participating in intervention

Percentage of total diet consumed at school

School-based intervention Scale up factor

Phyisical activity at school

Reduction In student canteen purchases

Percentage of children attending School

Sports voucher interventions

Sports vouchers Scale up factor

Percentage of the population that Registers for vouchers

Average Number of Hours of organised sports or activities Weekly

Average proportion attributed to vouchers

Average Metabolic Equivalent of Task (MET) for sports Inputs

Sports voucher interventions: Relapse Rate

Sugar-Sweetened Beverge Tax %

Sugar-Sweetened Beverge Scale up factor

Price Elasticity - Cordial

Price Elasticity - Fruit Drink

Price Elasticity - Soft drink

Sugar-Sweetened Beverge Tax: Relapse Rate

Sugar-Sweetened Beverge Tax %

## R version 4.3.2 (2023-10-31)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Pop!_OS 20.04 LTS
## 
## Matrix products: default
## BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 
## LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0
## 
## locale:
##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
##  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
## 
## time zone: Australia/Sydney
## tzcode source: system (glibc)
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] kableExtra_1.3.4 scales_1.2.1     ggpubr_0.6.0     ggh4x_0.2.5     
##  [5] patchwork_1.1.3  openxlsx_4.2.5.2 lubridate_1.9.2  forcats_1.0.0   
##  [9] stringr_1.5.0    dplyr_1.1.2      purrr_1.0.2      readr_2.1.4     
## [13] tidyr_1.3.0      tibble_3.2.1     ggplot2_3.4.3    tidyverse_2.0.0 
## [17] progress_1.2.2  
## 
## loaded via a namespace (and not attached):
##  [1] gtable_0.3.4      xfun_0.40         bslib_0.5.1       rstatix_0.7.2    
##  [5] tzdb_0.4.0        vctrs_0.6.3       tools_4.3.2       generics_0.1.3   
##  [9] fansi_1.0.4       highr_0.10        pkgconfig_2.0.3   webshot_0.5.5    
## [13] lifecycle_1.0.3   farver_2.1.1      compiler_4.3.2    munsell_0.5.0    
## [17] carData_3.0-5     htmltools_0.5.6   sass_0.4.7        yaml_2.3.7       
## [21] pillar_1.9.0      car_3.1-2         crayon_1.5.2      jquerylib_0.1.4  
## [25] cachem_1.0.8      abind_1.4-5       tidyselect_1.2.0  rvest_1.0.3      
## [29] zip_2.3.0         digest_0.6.33     stringi_1.7.12    bookdown_0.35.1  
## [33] labeling_0.4.2    rmdformats_1.0.4  fastmap_1.1.1     grid_4.3.2       
## [37] colorspace_2.1-0  cli_3.6.1         magrittr_2.0.3    utf8_1.2.3       
## [41] broom_1.0.5       withr_2.5.0       prettyunits_1.1.1 backports_1.4.1  
## [45] timechange_0.2.0  rmarkdown_2.24    httr_1.4.7        ggsignif_0.6.4   
## [49] hms_1.1.3         evaluate_0.21     knitr_1.44        viridisLite_0.4.2
## [53] rlang_1.1.1       Rcpp_1.0.11       glue_1.6.2        xml2_1.3.5       
## [57] svglite_2.1.1     rstudioapi_0.15.0 jsonlite_1.8.7    R6_2.5.1         
## [61] systemfonts_1.0.4